Goto

Collaborating Authors

 utility score


Modeling Contextual Passage Utility for Multihop Question Answering

Jain, Akriti, Garimella, Aparna

arXiv.org Artificial Intelligence

Multihop Question Answering (QA) requires systems to identify and synthesize information from multiple text passages. While most prior retrieval methods assist in identifying relevant passages for QA, further assessing the utility of the passages can help in removing redundant ones, which may otherwise add to noise and inaccuracies in the generated answers. Existing utility prediction approaches model passage utility independently, overlooking a critical aspect of multihop reasoning: the utility of a passage can be context-dependent, influenced by its relation to other passages - whether it provides complementary information or forms a crucial link in conjunction with others. In this paper, we propose a lightweight approach to model contextual passage utility, accounting for inter-passage dependencies. We fine-tune a small transformer-based model to predict passage utility scores for multihop QA. We leverage the reasoning traces from an advanced reasoning model to capture the order in which passages are used to answer a question and obtain synthetic training data. Through comprehensive experiments, we demonstrate that our utility-based scoring of retrieved passages leads to improved reranking and downstream QA performance compared to relevance-based reranking methods.


CoS: Towards Optimal Event Scheduling via Chain-of-Scheduling

Zhao, Yiming, Tang, Jiwei, Di, Shimin, Zheng, Libin, Yu, Jianxing, Yin, Jian

arXiv.org Artificial Intelligence

Recommending event schedules is a key issue in Event-based Social Networks (EBSNs) in order to maintain user activity. An effective recommendation is required to maximize the user's preference, subjecting to both time and geographical constraints. Existing methods face an inherent trade-off among efficiency, effectiveness, and generalization, due to the NP-hard nature of the problem. This paper proposes the Chain-of-Scheduling (CoS) framework, which activates the event scheduling capability of Large Language Models (LLMs) through a guided, efficient scheduling process. CoS enhances LLM by formulating the schedule task into three atomic stages, i.e., exploration, verification and integration. Then we enable the LLMs to generate CoS autonomously via Knowledge Distillation (KD). Experimental results show that CoS achieves near-theoretical optimal effectiveness with high efficiency on three real-world datasets in a interpretable manner. Moreover, it demonstrates strong zero-shot learning ability on out-of-domain data.


We thank all the reviewers for their careful readings and constructive comments

Neural Information Processing Systems

We thank all the reviewers for their careful readings and constructive comments. Note that following the RUM interpretation of MNL model (please see response to Rev. #3 for details), the score Re. Applications: As discussed in the Introduction, some motivating applications of our problem lies in various kind of Moreover, as we clarified in Rem. 1 and 2, for the special case of only two-sized subsets (i.e. when k = 2), our regret Bandits to subsetwise feedback ( Multi-Dueling bandits), also use the same notion of regret as ours (see Ref. [11,39]). We sincerely request the reviewers to kindly reconsider their scores based on the above clarifications. Mixed mnl models for discrete response.


Explanation-based Data Augmentation for Image Classification

Neural Information Processing Systems

Existing works have generated explanations for deep neural network decisions to provide insights into model behavior. We observe that these explanations can also be used to identify concepts that caused misclassifications. This allows us to understand the possible limitations of the dataset used to train the model, particularly the under-represented regions in the dataset.



Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions

Boeken, Philip, Zoeter, Onno, Mooij, Joris M.

arXiv.org Machine Learning

Performative predictions are forecasts which influence the outcomes they aim to predict, undermining the existence of correct forecasts and standard methods of elicitation and estimation. We show that conditioning forecasts on covariates that separate them from the outcome renders the target distribution forecast-invariant, guaranteeing well-posedness of the forecasting problem. However, even under this condition, classical proper scoring rules fail to elicit correct forecasts. We prove a general impossibility result and identify two solutions: (i) in decision-theoretic settings, elicitation of correct and incentive-compatible forecasts is possible if forecasts are separating; (ii) scoring with unbiased estimates of the divergence between the forecast and the induced distribution of the target variable yields correct forecasts. Applying these insights to parameter estimation, conditional forecasts and proper scoring rules enable performatively stable estimation of performatively correct parameters, resolving the issues raised by Perdomo et al. (2020). Our results expose fundamental limits of classical forecast evaluation and offer new tools for reliable and accurate forecasting in performative settings.



We thank reviewers for their constructive comments, please see below for our response

Neural Information Processing Systems

We thank reviewers for their constructive comments, please see below for our response. We will make this clear in the revised version. We will include the new results in the revision. Reviewer#2-1-Why SVT suffers from low accuracy. PC's original privacy guarantee might not hold because the sensitivity of the utility score calculated with greedy search We will make the statement more clear in the revision.



Towards Reliable AI in 6G: Detecting Concept Drift in Wireless Network

Tziouvaras, Athanasios, Fortuna, Carolina, Floros, George, Kolomvatsos, Kostas, Sarigiannidis, Panagiotis, Grobelnik, Marko, Bertalanič, Blaž

arXiv.org Artificial Intelligence

--AI-native 6G networks promise unprecedented automation and performance by embedding machine-learning models throughout the radio access and core segments of the network. However, the non-stationary nature of wireless environments due to infrastructure changes, user mobility, and emerging traffic patterns, induces concept drifts that can quickly degrade these model accuracies. Existing methods in general are very domain specific, or struggle with certain type of concept drift. In this paper, we introduce two unsupervised, model-agnostic, batch concept drift detectors. Both methods compute an expected-utility score to decide when concept drift occurred and if model retraining is warranted, without requiring ground-truth labels after deployment. We validate our framework on two real-world wireless use cases in outdoor fingerprinting for localization and for link-anomaly detection, and demonstrate that both methods are outperforming classical detectors such as ADWIN, DDM, CUSUM by 20-40 percentage points. Additionally, they achieve an F1-score of 0.94 and 1.00 in correctly triggering retraining alarm, thus reducing the false alarm rate by up to 20 percentage points compared to the best classical detectors. Cellular networks have undergone significant transformations since their inception, driven by the pursuit of higher performance, broader capabilities, and innovative services.